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ABSTRACT 

One approach to the problem of analyzing classroom 
interaction by use of a Markoff chain is presente . The two sections 
of the paper present (1) a discusion of Hoel’s test of order of a 
Markoff chain, and a Likelihood Ratio criterion (LRC) for a 
two-dependent Markoff chain is given, and (2) a discussion of 
possible adjustments of Darwin’s LRC with which to analyze data on a 
ona-dependent chain assumption. The results of Hoel's test of order 
of a Markoff chain showed that a two-dependent (Order two) model is 
better fit to interaction data than the one-dependent model (Order 
one) . Tables and graphs present the statistical data. (DB) 
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INTRODUCTION 



When a set of interaction data based on Flanders’ 
Interaction Category System, FLICS is recorded in a 
matrix , the purpose is to look at paired sequences of 
observation (recorded in the "cells” of the matrix) . 

The ten categories are given .in Table 1. 

Flanders [1] discussed the interpretation of various 
sections of the matrix which are combinations of the 
paired sequences. 

The concept of an interaction analysis matrix is 
closely related to a one-dependent Markoff chain (also 
simply called a Markoff chain) . Many researchers are 
analyzing differences between two or more interaction 
matrices with the use of a criterion based on a Markoff 
chain model . 

Feller’s [2, p. 340] definition will be paraphrased 
here and subsequently applied to interaction analysis 



data. 



A Markoff chai n is a sequence of trials witn possible 
outcomes E^ , in which the probabilities of sample 

sequences are defined by 

P. 



p(E. 5 E. * * • • >E } = a. p. . p. . 

3 0 3 1 3 n 3 0 Vl 3 1 3 2 



• ? 



^ j j * 1 J 

n-2 n-1 "n-1 n 



lr M T CS is an acronym adopted by the author, to refer 
specifically to Flanders’ ten-category system of record- 
ing verbal behavior, in a sequence, as they occur in a 
classroom--the coding taking place at three- second inter 
vals ; the data are used for interaction analysis. 
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TABLE 1 

CATEGORIES FOR INTERACTION ANALYSIS 



1. 
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8. 



W 

g 

CO 



9. 



Accepts Feeling : accepts and clarifies the feeling 

tone the students in a nonthreatening manner. 
Feelings may be positive or negative. Predicting 
or recalling feelings are included. 

Praises or Encourages ; praises or encourages stu- 
dent action or behavior. Jokes that release ten- 
sion, not at the expense of ano'her individual, 
nodding head or saying, "urn hm?" or "go on" are in- 
cluded. 

Accepts or Uses Ideas of Student ; clarifying, 
building, or developing ideas suggested by a stu- 
dent. As a teacher brings more of his own ideas 
into play, shift to category five. 

Asks Questions : asking a question about content or 

procedure with the intent that a student answer. 



Lecturing : giving facts or opinions about content 

or procedure; expressing his own ideas, asking 
rhetorical questions . 

Giving Directions ; directions , commands , or orders 
to which a student is expected to comply. 

Criticizing or Justifying Authority : statements 

intended to change student behavior from non- 
acceptable to acceptable pattern; bawling someone 
out; stating why the teacher is doing what he is 
doing; extreme self-reference. 



Student Talk — Response : a student makes a pre- 

dictable response to teacher. Teacher initiates 
the contact or solicits student statement and sets 
limits to what the student says . 

Student Talk — Initiation : talk by students which 

they initiate Unpredictable statements in re- 
sponse to teacher. Shift from 8 to 9 as student 
introduces own ideas. 



10. Silence or Confusion : pauses, short periods of 

silence and periods of confusion in which communi- 
cation cannot be understood by the observer. 
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in terms of an initial probability distribution a k 
for the states at tire 0 and fixed conditional 
probabilities of E , given that E. has occurred 

at the preceding trial. 

In interaction analysis this definition means 
essentially that anywhere in the interaction sequence, 
the probability that category (state) j will occur 
depends only on the preceding category i, and not on 
any other previous categories in the sequence. As an 
example, the probability that category nine (student 
talk--init iat ion) occurs in a classroom depends only 
on what occurred immediately before the nine. Look- 
ing at Table 1, one may guess that a nine is more 
likely (has a greater probability) to occur, if the 
preceding category is a three (accepting ideas of 
student) than if the preceding is a five. 

One approach to the problem of analyzing class- 
room interaction begins bv assuming that the sequence 
of observations is a one-dependent Markoff Cham. 

The categories are the ’’outcomes 5 * * ' ’^10 

the definition. The initial probability distribution 
a k is the probability of initial occurrence of any 
of the 10 categories. In a classroom, however, it is 
logical to assume that silence (or confusion) is 
always the initial state; this makes in the defini- 
tion equal to one for interaction analysis data. 
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The conditional probability p is the probability 
of occurrence of category k, given that j is the 
preceding category. 

Many researchers arc analysing differences be- 
tween two or more matrices for statistical signifi- 
cance by applying Darwin’s [3, p. 413]. Assuming a 
Markoff chain for interaction data, Darwin considered 
testing the Hypothesis that t sets 2 of values p.. 

^ij unknown, and i,j = l,...,s, where s equals the 
number of categories) are equal; that is, two or more 
matrices have the same for a given i and j . The 
data to which this refers are t matrices with long 
sequences. The Likelihood Ratio criterion to test 
the hypothesis is f 3 5 p. 413] 



£ 



2l2n. .. log n . . - Zn. .log n. - Zn. . + Zn. log n. ) 

JOh 1311 l.h * l.h 13. 1.. * 1.. 

( 1 ) 

distributed as chi square with 1,3 = l,...s, h = l,...,t, 
and s(t-l) (c-1) degrees of freedom. This criterion 
when applied to interaction data, was giving results 
which were too significant; that is, it was too 
sensitive to slight differences between sets of intc*r- 



* for those who refer to Darwin [3], the notation t 
is used 'instead of r, to avoid confusion with the use 
of r which denotes the order of a Markoff chain. 
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action data. The objectives in this inquiry were: 

1. to test the order of dependence 
of the interaction chain and, 

2. on the assumption of a one-dependent 
Markoff chain, 

a. to estimate empirically the 
power of Darwin's criterion 
as applied to two composite 
sequences, and 

b. to arrive at an application 
of Darwin’s criterion, which 
will reflect educational sig- 
nificance . 



This paper has two sections. In Section 
Hoel's [4] test of order of a Markoff chain will be 
discussed. The test showed that a two-dependent 
(Order two) model is a better fit to interaction data 
than the one-dependent model (Order one) — a model 
that is assumed when researchers use Darwin’s criterion. 
Hence, a Likelihood Ratio Criterion (LRC) for a two- 
dependent Markoff chain will also be presented. 

In Section II the author will discuss possible 
adjustments of Darwin’s LRC, if the researcher wishes 
to analyze data on a one-dependent chain assumption. 

In order to do this, a specific alternative 
hypothesis that two interaction sequences are not 
equal will be used to calculate the power of Darwin's 
criterion, given such inequality. That is, it will 
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be shown that, given two matrices which are not equal 
only by some chance and not due to differences in class 
room interaction, Darwin’s LRC will “reject" the null 
hypothesis of equality 100% of the time, when applied 
to 500 pairs of matrices generated from the two matrice 

From the generated pairs of matrices an empirical 
distribution of Darwin’s LRC will be derived in order 
to determine a cut-off point for rejection of the null 
hypothesis, such that rejection would have not only 
statistical significance but also educational signifi- 
cance. 

Sections I and II may be read independently of 
each other. 



Section I 

Hoel’s Test of Order of Dependence 
The data used in this section came from the data 
bank of Flanders [5]. These were the sixth grade inter 
action sequences of 30 classrooms on five different 
subject areas or activities. 

Hypotheses 

In general notation, the null hypotheses being 
tested are expressed as follows: 
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The transition probability, p_. , in an r-step 

— ,, . .. ■ 13 • • • K-L - — 

chain i s eq ual t o the transition probability , P, j , , .fci* 
in an (r - l)-step chain. Thus: 



H 0 : ^ii . . . kl " P.i«..kl 



1 “ 1 , 2 , • • 



A , 

In this paper we will let p.. = n /n 

* " Kl 13... hi lj... h. 

be the Maximum Likelihood estimator' of p.. . , • If the 

hypothesis 5 .s true, then the. Likelihood Ratio criterion, 

-2 lop X given below, 3 is, for large samples, distributed 
as a x 2 with s r ~ 1 (s - l) 2 degrees of freedom: 



X = -2 log X = 2 _ 1 . ^ “ij...kl 

1 , • • • ,1 



n. 

X n „*4 1,1 log ft 



ij . . .kl 



n 



. 3 . . .kl 



- loR n . 

il 3 . . . k . . 3 . . . k * 



( 2 ) 






In particular, the two null hypotheses tested to- 

2 

g ether with their appropriate x 1 s • 



(!) H : 



P .jk ’ 



i = 1 , ... ,10 




3 The logarithms in the formulas throughout this paper 
are all to the base e. 
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where 



- Ri 3 k 

fijk “ riij. 



^ n * * i ^ ^ 

2 n k , „ O k \ 

X (2) “ 2 .** "ijk (1 °S ST" * 105 “• ) 



l i3 



( 3 ) 



( 2) H q : p.. = P.. 



where 



P 



ID 



n . . 

3-D 

n • 



n . . n . 

X* = 2 Z n,. , (log - log ~1 
UJ M ij n • n.. 

13 1 * 



(4) 



The x 2 subscript refers to the order of dependence r, 
being tested against an (r - 1) -dependence , and, from what 
follows, to say that X 2 (r) is not significant means that an 
(r - 1) -dependence is as good as an r-dependence assumption 
Therefore, an (r - 1) -dependence would be preferable. 

The basic idea behind Hoel's test is that a suffi- 
ciently large r is chosen and testing is done successively, 
decreasing r by one each time until a point is reached 
when r is not significant and r - 1 is significant, lhis 
is so because, if a chain is Markovian and the length of 
the dependence is r + 1 (i.e., the chain is r step and the 
dependence extends over r + 1 consecutive variables) , 




4 



9 



9 



^ (r) > X • • • will not bo significant in increasing 

o levels, while X( r _i)> x‘( r _ 2 ) ••• X* will be signifi- 

cant, in decreasing a levels. 

The data did not reasonably allow a test beyond r = 2. 

The tost was made, not so much to locate the order of 

dependence as to have a basis of choice between an r ~ 2 

and an r = 1 assumption. The result of Hoel’s test would 

2 

be conclusive as to best fit only if x would not 

A 

turn out to bo significant while was significant. 

In this case it would mean that an (r-1) dependence assump- 
tion would suffice, while an (r-2) dependence wouldn’t. 
However, if both r and (r-1) dependence were significant, 
the judgment as to whether an r-dependent model was a better 
fit than an (r-1) dependence would be discretionary, after 
the magnitude of difference in the a levels was observed. 

Application of the Test 

Hoel’s test was run on each cf the matrices of the 
30 teachers, as well as on each of the five activities. 

The matrices of the activities wore formed as follows: 

Five teachers were choc»cn at random and their interaction 
data separated into five sequences* (five matrices), one for 
each of five activities in which they were observed. All 
five matrices of each activity were combined, yielding one 
matrix for each activity. The iiv^ activities were 
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adminis* : ative routine, language arts, social studies, 
mathematics, and science. 

Results of Koel's Test 

2 

Table 2 summarises the results of the x tests on 
each teacher's matrix, anci Table 3 those on each ox the 
five- activities. The z-column in Table 2 shows that all 

the 3 0 x’ f s were significant with a < .001 for twenty nine 
teachers and a < .01 for one teacher (Teacher 19). The 
z-column in Table 3 shows that for r = 2 the adminis- 
trative routine and science chains (across the same five 
teachers in the other activity chains) wore not signifi- 
cant. The results on these two activities (administra- 
tive routine and science) may not be true reflections of 
the actual length of dependence because the table shows 
that the two chains were relatively much shorter than 
those of the three other activities. 

n 

The z values (s = /2x - /2df-l; see footnote of 

Table 2) for r = 2 for the 30 teachers range from 3.24 
to 22.67, while those for r = 1 range from 77.4 to 130.23. 

Hence, a one-dependent chain was still a bad fit and aero 
dependence should definitely not be considered at all. 

The marked decrease in z from the assumption of one- 
dependence to the assumption of two-ciepcndence , may indi-; 

Vr 

cate that had it been possible to test the chain for w = 3 
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TABLE 2 

TEST OF ORDER OF MARKOFr CHAIN ON 30 TEACHERS 



Teacher 

No* 


Length of Chain 
(No. of Tallies) r 


X 2 


d£ . 


Z 3 


00 


6019 


2 


1415.11 


810 


12.96 






1 


5986.71 


81 


96.73 


03 


8388 


2 


1618.01 


810 


16.65 






1 


10213.53 


81 


130.23 


12 


6265 


2 


1516.75 


810 


14.84 






1 


6085,61 


81 


97.63 


13 


5781 


2 


1306.63 


810 


10.88 






1 


7162.28 


81 


107.00 


15 


7080 


2 


1476.84 


810 


14.11 






1 


7618. 80 


81 


110 i 75 


19 


6858 


2 


945.10 


810 


3.24 






1 


8413.77 


81 


117.03 


24 


8117 


2 


1826.22 


810 


20 ,20 






1 


8042,57 


81 


114,14 


26 


6323 


2 


1339.10 


810 


11,51 






1 


5814* .53 


81 


95,15 


27 


6557 


2 


1613.09 


810 


16 . 56 






1 


5733.54 


81 


94.40 


28 


8360 


2 


1718.08 


810 


18.38 






1 


9398.20 


81 


124.41 


30 


7886 


2 


1239.17 


810 


9.55 






1 


7478.43 


81 


109.61 


34 


6339 


2 


1972.84 


810 


22.58 






1 


4639.90 


81 


83.64 


37 


6386 


2 


1567.70 


810 


15.76 






1 


4070.33 


81 


77.54 


40 


7216 


2 


1565.61 


810 


15.72 






1 


6492.02 


81 


101.26 


42 


6175 


2 


1538.02 


810 


15.23 






1 


5854.98 


81 


95.52 
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TABLE 2 — 


Continued 










Teacher 

No. 


Length of Chain 
(No. of Tallies) 


2 

X 


df 


z a 


48 


7103 


2 


1298.55 


810 


10.72 






1 


6839.02 


81 


104.26 


50 


7233 


2 


1403.73 


810 


12.75 






1 


5958.53 


81 


96.48 


51 


8884 


2 


1593.88 


810 


16,22 






1 


9294.03 


81 


123.65 


53 


7462 


2 


1129,70 


810 


7,30 






1 


5140.46 


81 


88.70 


54 


7903 


2 


1734.55 


810 


18,66 






1 


6315.20 


81 


99,70 


64 


6544 


2 


1279,30 


810 


10,35 






1 


6478 , 09 


81 


101.14 


72 


6831 


2 


1748.34 


810 


18.90 






1 


6725.48 


81 


103.30 


73 


6086 


2 


1313,20 


810 


11,01 






1 


4216,91 


81 


79,15 


75 


5586 


2 


1104,47 


810 


6.76 






1 


6557.90 


81 


101.84 


77 


7201 


2 


1376 o 54 


810 


12,23 


■ 




1 


5510,03 


81 


92,29 


80 


9075 


2 


1978.74 


810 


22.67 






1 


7030,50 


81 


105,89 


84 


6639 


2 


1454.74 


810 


13,70 






1 


7932.27 


81 


113,27 


89 


6789 


2 


1483,50 


810 


14.23 






1 


7829.44 


81 


112.45 


91 


5749 


2 


1391.83 


810 


12.52 






1 


5068,90 


81 


88.00 


95 


6339 


2 


1698.90 


810 


18.05 






1 


4870.74 


81 


86.01 




a Because d.f. is large (> 70), the expression 

2 » - /2d“ T! - 1 was used as a normal deviate with 

unit variance, whereby the probability for x 2 corresponds 
with that of a single tail of the nor^J curve. 



13 



against r = 2, the z values may not have turned out to be 
significant, indicating that eh*. best fit on the Markoff 
chain assumption v/ould be: the two-dependent chain model. 



TABLE 3 

TEST OF ORDER OF MARKOFF CHAIN ON FIVE ACTIVITIES 



Chain* 

Description 


Length 
of Chain 


r 


x 2 


df 


z 


Adm. routine 


2398 


2 


703.17 


810 


-2.74^ 






3. 


2606.54 


81 


59.51 


Lang, arts 


11756 


2 


3009.29 


810 


37.34 






1 


12507.93 


81 


145.48 


Soc. Studies 


7262 


2 


1882.40 


810 


21.12 






1 


7233.41 


81 


107.59 


Mathematics 


8547 


2 


1487.62 


810 


14.31 






1 


7348.40 


81 


108.54 


Science 


2506 


2 


686 . 54 


810 


-3.18 b 






1 


2971.44 


81 


64.40 


a The chain 


is across 


five 


teachers 


( same 


activity) 


drawn at random from the 


30 t< 


aachers in 


the i 


study. 



b Mot significant. 
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A Likelihood Ratio Statistic 
for the Two-Step Model 

This part of Section I is mainly expository. 

It presents a simple application of the Likelihood Ratio 
test for a Markoff chain of order two. Such a test 
appears in the literature on the subject. For example, 
Anderson and Goodman [8, p. 103] obtained a x test of 
goodness of fit, i.o., a test of the hypothesis that 
two samples are from the came Markoff chain of a given 
order. 



Maximum Likelihood (ML) Estimato r of 

Under the two-dependent chain model, the ML 
estimator of p is [4, p.430]: 





IS 







( 5 ) 



If there are t matrices to be compared, (5) i s the 
ML estimator of the p. .. of each of the t matrices. 

1 j K 

i. u 

for the h matrix, the ML estimator is 



Hence , 



n . ■ . « 
1 jkh 

^ijkh * “ 

n ij.h 



( 6 ) 



An Estimator of Under the Hypothesis 

That t Matrices Are Equal 



The test statistic to be developed is a test of the 

hypothesis (Hq) that two or more matrices of interaction 

data under the two-dependent Markoff chain model are equal. 

Let be the probability derived from some known 

distribution of the n^^h representing the probability for 
th 

the h teacher to have an observed interaction fall on 
fch 

the k category (state) , given that two preceding states 
are i , j . Then 



V p 



ijkl P ijk2 



= P 



ijkt* 



(7) 



Under the assumption that the interaction data for 
each teacher are generated by the same two-step Markoff 
process, the hypothesis in (7) states that the transi- 
tion probability is the same for each teacher. 

Hence, the estimator of P^j kh under H Q can be equated 



16 






16 



with the Maximum Likelihood estimator of 
step Markoff chain, that is. 



p. of a two - 
‘ ijk 



P. ... 
13 kh 



ay 




( 8 ) 



The hypothesis H in (7) states that the t sets 

of p.. are the same for everv combination i, j. There 
13 kh 

are , therefore tables (s x t) of the contingency type. 
Under this hypothesis, 

A 

Pijkh< K 0 ) s n ijk. /n ij.. • (s) 

Equation (9) is an estimator for Pj» k for any of 
the t matrices under the hypothesis set up in (7). 

(uf) 

The ratio of the Likelihood Function^ evaluated for the 

estimator in (9) to that evaluated for the ML estimator 

of p.,,, in (6) is the Likelihood Ratio . If K of 
r iikh t 

(7) is true, this Likelihood Ratio will not be signifi- 
cantly different from one . 



Development 

KaTTo 



of the Likelihood 
Statistic *”* 



The LF of p 



• 1 . . 

1*1 kn 



is 



L 





n 



iikh 



( 10 ) 



Let be the ML estimator of and P^j^CO) be 

the estimator under H fyj© have the following: 






'f *V 
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n . . 



^i-ikh = n " < for each h) • 
i]kh n. j>h 



(ID 



Pijkh(O) ^ijk n^’ (for any h) . 

X J • • 



( 12 ) 



X = 



_ V max ) = L ^iikh(0)) 

L (max) I '^iikh ) 



(13) 



. n.., n ijkh 

Is TT ( 

ijkh(0)J 



(14) 



L ^i-ikh } = ” (n^) 

ijkh n ij.h 



n IJtl . n ijkh 



(IS) 



iog X - . E . n ijkh (l°g ^ - log . 

lj ♦ • 



ijkh 



n. . . 

ij .h 



( 16 ) 



-2 



109 X = 2( ijkh ni i kh 109 n li kh “ J kh "ijkh log n ij.h 



jkh 



ijkh 

£ n 
ijkh 



jkh 



ijkh 



I 

4 -iVU 


n . .. . 
ijkh 


109 n ijk. 


+ £n 


ijkh 109 


n. . 

13 •• 














(17) 


mmations of 


equation 


(17) 


are 




log 


n . . . « 
13 .h 


£ 

ijh 


"ij.h 


log 


n i+ V\ 

l j • n 


(18) 


log 


n . . , = 

13 k. 


£ 

ijk 


n ijk. 


log 


n. .. 
13 k. 


(19) 



and 
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2 n. .. . log n. . = £ n. . log n. . 

ijkh 13kh 13 •• ij • • y • • 



( 20 ) 



the dot taking the place of the summation sign. 



Substituting (18) , (19) , and (20) in (17) , 



we have 



“2 log X — 2( £ n i-iich n i -iich ^ n. . . log n. . 

ijkh 13kh 13kh ijh 13 * h 13 • 



" 2 "i-ik lo 9 n --ik + J X°g n. . ) 

lj«« 1 J • • ^ 



( 21 ) 

Eq. ( 21) is the LR test criterion of difference 
among t matrices, on the assumption of a Markoff chain of 
order two. Each matrix is identified by the h subscript, 
h = l,...,t, and n^.^ denotes the frequency of the sequence 
x <n-2t) = i( X (n-x) = ^ x (n) = kj £or teacher h> The dot 

means that summation has been carried out over the replaced 
subscript, so that 



n « ■ * ■“ £ n • i • ■ « 
iD.h ^ i;)kh' 



n ijk.“ l n ijkh' 



and n. . = £ n. . 

13 •• kh i3kh 



It has been shown in literature that -2 log X (3.371 

2 2 

is distributed asymptotically as x * on s (s-1) (t-1) 
degrees of freedom. 
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Section II 

Analysis of One-Dep e ndent (Interaction) Chains 

The Problem of Comparing Two 
Interaction Sequences 

When interaction data are displayed in a 10 x 10 
matrix form, the underlying assumption is that these data 
were generated by a Markoff chain of order one. In order 
to test the equality of two interaction analysis sequences;, 
one computes Darwin's LP.C (Eq. (1) ), 

2[rn jkl lo S n jkl - Jn j.l lo 5 n j.l Xn jk. lo S n jk. 

+ En. log n. ] , 

j • • 3 • • 

which has a Chi Square distribution for a large n. Then 
using the Chi Square tabic:, he determines the statistical 
significance of the difference between the two sets of data. 

The results of past studios, however, have shown that 
the test is so sensitive that small differences between two 
interaction sequences yield a significant Chi Square. Hence . 
to interpret the results of the test from an educational 
viewpoint- -that is, to see whether the statistical signi- 
ficance has any practical meaning in education--one needs 
to find out how often the test rejects the null hypothesis 
of equality when it is assumed that N pairs of sequences to 
which Darwin f s LP.C is applied are generated by a pair of 
Markoff chain models with known transition probabilities and 
when it is assumed that the transition probabilities indicate 
that the sequences in the pair are educationally homogeneous. 

The proportion of the times the test would reject 
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the null hypothesis of equality would be an approximate 
measure of the chance of being right, in the statistical 
sense, in inferring that the two interaction sequences 
are different from each other. Since Darwin’s test is 
very powerful, it is possible that all Chi Square tests 
would reject H Q , i.c. , the power of the test equals one . 
Therefore, one would quite frequently make a statistical 
inference that the two sets of data, of the type speci- 
fied, are significantly different. However, since the two 
sample sequences were generated from two educationally 
homogeneous matrices, most of the time our inference is 
wrong in the educational sense. Hence, the interpreta- 
tion of the statistical significance of Darwin's LRC can 
be very misleading, especially if one ha*s to make an 
inference from such a test that one type of teacher be- 
havior is more effective in producing certain educational 
outcomes than is another. One way to estimate the power 
of the test would be to study an empirical distribution 
of N Chi Squares and see how each Chi Square value is 
associated with an estimated probability of occurrence. 

In this way, one may evaluate a computed Chi Square in 
terms of the particular distribution. 

In this section, (1) an estimate of the power of 
Darwin's criterion will be obtained and (2) a cumulative 
probability distribution of the Chi Square values from 
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pairs of sequences generated from two educationally 
homogeneous sequences through computer simulation. 

The LRC and the Length of Sequence 
When two interaction sequences are compared, an 
estimate of the power of Darwin’s test is not the only 
problem to arise. A secondary problem is an approximation 
of what happens to the LRC as the length of the sequence 
(number of tallies) increases. It can be observed that 
the size of LRC increases with an increase in the number 
of tallies. The increase in the size of the LRC, in this 
example, docs not relate to the degrees of freedom which, 
in turn, do not depend on che length of the chain (or 
number of tallies) but on the number of categories (s - 10) 
and the number of sequences b^ing compared — in this problem, 
two sequences. The degrees of freedom remain constant for 
a given category system and for a specified number of 
sequences to bo compered. This means that one factor 
influencing the outcome of an LRC test is the length of 
the interaction sequences chosen for the comparison. 

Computer Simulation 

The Choice of H^: ( p jkl^ * (Pjk2^* The °f 

parentheses indicates that P.^ and Pjk2 matrices of 
transition probabilities. For convenience, -he researcher 
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deviated slightly from the subscript notation used in the 
Section I and shifted to Darwin's. 

In terins of educational significance , two interaction 
sequen ces , display ed in matrix form as (n^ and ■> 

arc said to be education ally homogeneou s if the two c 1 a 3 s - 
room situation s which they re present have the sane educa - 
tion cal outcomes on th e basis of an outside criterion , c » g ♦ ; 
achievement or at titude . Here, homogeneity refers to edu- 
cational outcome and is to be distinguished from equal tran- 
sition probabilities . To obtain a pair of unequal sequencer 
for H^ } two identical sequences may bo made to vary a. little 
by slightly changing the frequency in one cell <or the 



frequencies in a few cells) of the matrix. One would 
judge, without further tests, that the two sets of class- 
room interaction which th^ two matrices represent were 
extremely homogeneous. One could continue modifying cell 
frequencies and still produce two educationally homogene- 
ous sequences. For the results of this study to be mean- 
ingful, one should choose a pair which would satisfy the 
criterion of being representative of a. real situation, 
one v:her^ two teachers arc producing the same educational 
outcomes through similar classroom situations. 

The first question this section attempts to answer 
is: How sensitive is bargain's test to differences between 

two classroom interaction sequences? That is, how often 
does Darwin’s test tend to reject the hypothesis that two 
sequences are equal when it is known that they are 
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homogeneous in terms of educational outcomes? 

Thu answer to these questions can be approximated 
by choosing a pair of sequences v.’i'th optimum inequality. 
That is, the two sequences should represent a. pair of 
realistic classroom situations which are judged to be 
educationally homogeneous. Those. two sequences would be 
made to generate pairs of sequences to be tested for 
equality by Darwin’s LRC and the percentage of rejection 
would be observed. This percentage is the proportion of 
times one would make a "Type I error" in the educat ional 
sense , that is, the proportion of tiiris.s one would reject 
the hypothesis of educational homogeneity when it is true 

In connection with the problem of choosing 9 two 
ideas should be recalled: (1) the underlying objective 

in this section was to find a way of interpreting any 
statistical significance in the light of educational 
significance : (2) the problem in testing differences be- 
tween two interaction data sets was the high sensitivity 
of the LRC ( statistically significant differences 
suited from ihe test , even though the two sequences being 
tested Were known to be practically the same) . 

The author created two composite matrices that 
wore based on typical transition probabilities, and 
in addition, represented homogeneous outcomes. If 
pairs of interaction sequences wwrc generated from this 
pair, one would get a distribution of Chi Square values 
and could see how often the hypothesis of equality would 
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be rejected. 

Criteria, for selection . The following statements 
summarize the criteria for choosing the pair of sequences 
for the alternative hypothesis: 

1. The two sequences have no signif icantly differ- 



ent educational outcomes. 



2. The proportions in the cells of the matrices 
represent some identifiable target population. To achieve 
these proportions, the researcher used actual data, instead 
of determining the extent of homogeneity by subjective 
judgment . 



3. The length of each sequence is realistic. 
Normally, 30 minutes to two hours of classroom observa- 
tion were, made in projects involving interaction analysis 
Source of data for the alternative hypothesis . 
Flanders [5] observed 16 eighth-grade mathematics classes 



The teachers all taught the same two-week unit of study, 



the materials of instruction being kept constant. Teacher 

influence was controlled by measuring the spontaneous 

patterns of teachers, while the adjusted final achievement 

scores of the students was an outcome variable . By 

"adjusted final achievement scores" is meant the scores 

which took into .account the initial ability of the students. 

# 

S teps in the C o mputer Simulation . 

The following is an outline, of the steps in generating 
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pairs of sequences and obtaining a probability distribu- 
tion of Darwin’s LRC. 

1. The 16 eighth-grade mathematics teachers were 
first ranked according to the adjusted post-test score. 
Then the odd-numbered teachers in the list formed one 
group, and the rest formed the other group. Table 5.1 
shows these two groups, with their adjusted post-test 
scores. The result was a nonsignificant difference 
between the mean achievement scores of the two groups. 
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TABLE 4 

I 



TWO 


GROUPS OF 16 EIGHTH 
HOMOGENEOUS 


GRADE MATHEMATICS 
ON ACHIEVEMENT 


TEACHERS , 




Group I 


Group 


II 


Code No. 


Achievement 

Score 


Code No. 


Achievement 

Score 


V804 


34.1 


C801 


30.7 


P805 


30.2 


D804 


29.8 


V803 


29.1 


M802 


28.9 


A802 


27.9 


M805 


27.3 


1801 


27.1 


T802 


26.5 


H802 


26.3 


A803 


26.2 


E802 


24.7 


G802 


24.1 


L803 


23.6 


G801 


21.3 


Means 


27.9 




26.9 



2. The matrices of the teachers in each group were 
then combined to form two composite matrices of the pair- 
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one composite matrix for each group. The use of several 
matrices combined, rather than selecting a single matrix 
of a short sequence-, would make the composite matrix more 
representative of a group of similar classroom situations. 
In other words, the ML estimates in a composite matrix 
approximate the averages of the separate ML estimates 
in the component single matrices. 



3. From the two composite matrices, the ML 
estimate of was computed. The ML estimate 

of p -jkl is 







, 10 , 1 = 1 , 2 . 



( 22 ) 



The estimates p,.^ and p.^ f ~ re £i ven in Table 5. 

It can be observed that the corresponding transi- 
tion probabilities of the two matrices (A) and (B) 
are not very different. 

4. Cumulative transition probabilities were 
computed from the ML estimates. These are shown 
in Tables 6 and 7. 

5. From the cumulative probabilities obtained in 
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TABLE 5 



MAXIMUM LIKELIHOOD ESTIMATES (TIMES 1000) OF TRANSITION 
PROBABILITIES pjfc FOR THE TWO SEQUENCES A AND B 
OF THE ALTERNATIVE HYPOTHESIS 



• 

1 


• 


1 


2 


3 


4 


5 


t -n Tsamc 
6 


7 


8 


9 


10 


1 


A 


097 


001 


020 


157 


373 


078 


039 


039 


059 


137 




B 


077 


015 


023 


101 


628 


001 


078 


001 


031 


047 


2 


A 


002 


046 


081 


167 


262 


095 


002 


015 


100 


229 




B 


003 


122 


143 


162 


281 


027 


016 


014 


09 7 


135 


3 


A 


002 


036 


304 


155 


304 


030 


007 


042 


073 


049 




B 


003 


023 


323 


214 


249 


015 


007 


027 


075 


064 


4 


A 


002 


004 


003 


155 


051 


024 


008 


635 


031 


086 




B 


001 


003 


004 


141 


040 


017 


004 


663 


0S4 


064 


5 


A 


001 


003 


003 


083 


797 


038 


013 


006 


027 


028 




B 


001 


002 


003 


103 


787 


025 


008 


003 


039 


027 


6 


A 


001 


002 


001 


048 


122 


323 


022 


219 


057 


205 




B 


001 


001 


002 


060 


122 


296 


047 


170 


084 


218 


7 


A 


001 


003 


002 


092 


206 


065 


388 


033 


047 


163 




B 


001 


005 


005 


066 


109 


101 


388 


024 


092 


208 


8 


A 


001 


045 


112 


139 


239 


068 


031 


313 


028 


024 




B 


002 


048 


214 


235 


208 


065 


015 


153 


027 


034 


9 


A 


002 


041 


165 


071 


247 


085 


045 


007 


278 


058 




B 


004 


032 


192 


075 


224 


064 


036 


002 


315 


055 


10 


A 


002 


Oil 


003 


074 


113 


098 


035 


033 


090 


542 




B 


001 


006 


005 


070 


099 


078 


058 


017 


111 


555 
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TABLE 6 

CUMULATIVE PROBABILITIES (TIMES 1000) 
FOR SEQUENCE A 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


097 


098 


118 


275 


647 


725 


765 


804 


863 


1000 


2 


002 


048 


129 


296 


558 


653 


655 


670 


771 


1000 


3 


002 


037 


341 


496 


799 


829 


836 


878 


951 


1000 


4 


002 


006 


009 


164 


215 


239 


247 


882 


914 


1000 


5 


001 


005 


008 


091 


888 


926 


939 


945 


972 


1000 


6 


001 


003 


004 


052 


174 


497 


518 


738 


795 


1000 


7 


001 


004 


006 


098 


304 


368 


757 


790 


837 


1000 


8 


001 


046 


158 


297 


536 


605 


635 


948 


976 


1000 


9 


002 


043 


209 


280 


526 


612 


657 


663 


942 


1000 


10 


002 


012 


015 


089 


202 


300 


335 


368 


458 


1000 







CUMULATIVE 


TABLE 7 

! PROBABILITIES 
FOR SEQUENCE 


• 

(TIMES 1000) 
B 






1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 


077 


091 


114 


215 


843 


844 


921 


922 


953 


1000 


2 


003 


124 


268 


430 


711 


738 


754 


768 


865 


1000 


3 


003 


026 


348 


562 


811 


826 


834 


861 


936 


1000 


4 


001 


004 


008 


150 


189 


206 


210 


873 


936 


1000 


5 


001 


004 


007 


110 


898 


922 


931 


934 


973 


1000 


6 


001 


002 


003 


063 


185 


482 


529 


698 


782 


1000 


7 


001 


006 


Oil 


077 


186 


287 


675 


700 


792 


1000 


8 


002 


050 


263 


499 


707 


772 


787 


940 


966 


1000 


9 


004 


036 


228 


304 


528 


592 


628 


630 


945 


1000 


10 


001 


007 


012 


081 


181 


259 


317 


334 


445 


1000 
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step four, 500 pairs of sequences wore generated. At 
the same time, the LRC and its standard deviate z were 
computed for each generated pair. The procedure for 
simulation is simply starting a chain with the category 
on silence, 10. A random number with uniform distribu- 
tion over the range 0.000 to 1.000 is generated, and 
the column category in row ten , with a probability 
greater than or equal to this random number, determines 
the next category, j. Thus, n 1Q . is incremented by 
one, i.c. , a tally is made in the (10, j) cell. This 
now category determines the next row, i (equal to the 
preceding j). A random number is again obtained to deter- 
mine the category (j) entry in this row. This cycle 
continues until the desired length of the sequence is 
reached. For each pair generated, the LRC and z values 
were computed. The 500 pairs generated produced 500 LRC 
and z values. 



5. The 500 LRC values were then ranked from small- 
est to largest in order to locate every fifth percentile 
for purposes of analysis. 

7 . Since the LRC is sensitive to the lengths of the 
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st.quc.ncx s in th«u pair-, data wurc also generated for differ- 
ent sequence lengths. In this step, only 20 pairs were 
generated for each length. The lengths tried were 500, 
1000, 2000, and G000 tallies, in addition to the length of 
4000 which had been used in generating the 500 pairs. 

8. The 20 LPC values in each set were then ranked 
from the smallest to largest to identify the percentile 
value for each LRC value. 

Results and Discussion 

Pairs of Sequences of Length 4000 . Table 8 is a 
cumulative distribution of the 500 values of Darwin’s 
LRC computed from the 500 pairs of sequences generated 
from the two composite matrices in stop two of the pre- 
ceding subsections. The lowest z value in thv? genera- 
tion is 3.622, which is still statistically significant 
(p < 0.01). Hence, if the sample pair being tested is 
the same type as the original pair- -eighth-grade data, 
homogeneous on achievement, etc. — in most (or all) cases, 
one would wrongly infer (wrong in the educational sense) 
that the two sccuenccs in the pair are not equal, i.e., 
not educationally homogeneous. 

It has now been demonstrated that, using two educa- 
tionally homogeneous sequences, Darwin’s test is so power- 
ful that the power is equal to one . For such time as 
researchers continue to use this test, the investigator 
suggests reducing the. probability of the Type I error 
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TABLE 8 



EMPIRICAL CUMULATIVE DISTRIBUTION 3 OF DARWIN’S 
LIKELIHOOD RATIO CRITERION AND 
ITS STANDARD DEVIATE Z 



Generation 
Pair No. 


LRCq 


1 

N 

O 


P ( 2 > Z Q ) 


304 


164.625 


4.755 


.95 


122 


172.445 


5.181 


.90 


23 


179.937 


5.580 


.85 


291 


183.641 


5,775 


.80 


118 


187.742 


5.987 


,75 


14 


192.312 


6,222 


.70 


436 


195 . 266 


6,372 


.65 


326 


199.102 


6.565 


.60 


461 


201.586 


6.689 


,55 


359 


204*281 


6,823 


.50 


114 


206,391 


6,927 


.45 


378 


209,930 


7.100 


,40 


482 


212 o977 


7 e 249 


,35 


19 7 


217.008 


7.443 


.30 


130 


222,016 


7.682 


,25 


266 


225.617 


7.852 


.20 


271 


230.859 


8.0-98 


.15 


221 


235.672 


8 , 320 


.10 


131 


247.414 


8.855 


.05 


240 


250,445 


8.991 


.04 


223 


253.344 


9.120 


.03 


488 


257.172 


9.289 


.02 


8 


261. 375 


9.474 


.01 


104 


274.680 


10,048 


.00 



a The length of a sequence in the 500 pairs 
of sequences generated, is 4000 tallies. 
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in order not to reject a large number of hypotheses which, 
though statistically false, are educationally true. In 
other words, the cutoff point in the range of LRC values 
should be much greater than thos^ values at the usual 
levels of significance, i.e., greater than the Chi Square 
values at .05, or .01, or'lGQl. In effect, one reduces 
the power of the test even as far as .05. 

Thus, from Table 8, under the given alternot ive 
hypothesis , 

p ( 2 > 8.855) = .05. (23) 

That is, if the cutoff point is set at z = 8.855, the 
power is reduced to approximately .05. The value .05 in 
the table represents the fifth percentile, or the proba- 
bility of rejecting the hypothesis of equality und^r the 
given alternative hypothesis. This probability is the 
empirical pother of trie t est , at z = 8.855 , under tire 
alternative hypothesis given by the transition probabili - 
ties in Table 5 . 

In contrast , the table of the normal probability 

7 

integral— single tail probability [i-6-] — gives 

-6 

p(z > 4.99) = 0.30190(10 ). (24) 

The above equation is based on the theoretical distribution 
of z under the hypothesis of equality of the two sequences, 
lienee, in the statistical sense, choosing z = 8.855 as the 
boundary of the critical region reduces to near-zero the 
level of significance a, or the probability of a Type I 
error . 
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In summary, the cutoff point of z in rejecting 
equality can be given by a normal distribution table, but 
the cutoff point for rejecting educational homogeneity 
would be suggested by a table such as Tabic 8. 

Figure 1 is a graph of the cumulative empirical 
distribution of z obtained from Table S. Under the 
alternative hypothesis of educational homogeneity given 
by Table 5 , the probability of a z value smaller than 
10 is near one. The application of the empirical dis- 



tribution of LRC values obtained at this stage would be 
limited, considering the number of assumptions to be met- - 
such cassumptions as length of sequence , type of class- 
room, grade level, subject taught, and educational 
criterion. However, The values in Table 8 would easily 
suggest the magnitude of the LRC which one might set as 
the boundary of the critical region. For purposes of 
illustration, suppose these assumptions were met by two 
sequences whose difference was being tested for signifi- 



cance. Suppose, the LRC obtained were 8.33. While a 



normal probability table indicates this value to be highly 
significant with a practically zero, Table 8. indicates 
that rejecting the hypothesis at this point leaves a high 
probability that the sequences are homogeneous from an 



educationa.1 standpoint. 

Pairs of S eque nces of Varying Lengths . Table 9 shows 
the ranked values of LRC and z computed from generated pairs 
of varying lengths- -20 pairs to each given length. For any 
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p(z < z 0 ) 




Fig. 1 — Cumulative empirical distribution of Darwin's 
Likelihood Ratio criterion computed from 500 pairs of 
sequences, generated from a pair of interaction sequences.* 

♦Length of each sequence = 4000 tallies. 
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given percentile rank, the increasing values of LRC for 
increasing lengths can be not^u. . Thus, a z of 4.98 would 
have a significant meaning in education if the number of 
tallies in each member of the pair is near 2000, but not if 
the number is near 4000, which requires a z of 9.17 for thv. 
difference: to have any significance in education. 



A Compariso n of the Empirical Distribution of Darwin's 

LRC with the x ' 2 Distribution . Ax 2 table gives values at 

90 degrees of freedom at certain percentile points. These 
2 

X values are posted in the first column of Table 10. The. 
last column gives the percentile points available from a 

X table. The remaining columns give the corresponding 



LRC values obtained from the generated sequences of lengths 
varying from 500 to GOOD. The table shows that the 
sequence length of 500 has LRC values closest to the tabu- 
lated x 2 values, with discrepancies widening as the criti- 



cal region (probability of a greater value) decreases. 
However, since the LRC for length 500 is close to but does 

0 O 

not significantly exceed x* 5 the tabulated values of 
could provide a safe guideline for educational inference 
for sequences of 500 tallies to each member of a pair. 

This statement is particularly true if the major concern is 
to minimize the chances of making a Type I error, that is, 
when one desires a greater margin of safety in being right 
in rejecting equality. 

When these values were plotted on the graph (Figure 2) 
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Graphical relationship between Tabulated X 2 (df = 90) values and LRC obtained from 
generated pairs of sequences of different lengths at given percentile po\nt9 
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A COMPARISON OF THE CUMULATIVE DISTRIBUTION OF x 2 90 FR0M A 



CHI SQUARE TABLE AND THE EMPIRICAL CUMULATIVE 
DISTRIBUTION OF DARWIN’S LRC COMPUTED 
FROM DIFFERENT SEQUENCE LENGTHS 



Darwin’s LRC Probability 



90 ~ 

(Table) 


Length 
= 500 


Length 
= 1000 


Length 
= 2000 


Length 
= 4000 


Length 
= 6000 


Greater 

Value 


69.13 


6 8.41 


87.21 


96.53 


164.63 


190.25 


.95 


73.29 


73.79 


92.33 


106.33 


172.45 


226.37 


.90 


80.62 


77.85 


96.05 


127.59 


187.74 


247.75 


.7 5 


89.33 


84.34 


101.41 


133.65 


204.28 


256.12 


.50 


98.64 


89.80 


108.75 


148.43 


222. 02 


267.00 


.25 


107.56 


100.62 


116.92 


160.55 


235. 67 


271.88 


.10 


113.14 


100.74 


131.02 


168.75 


247.41 


312.75 


. 05 


124.12 


102.27* 


140.27* 


179.48* 


261.38 


327.12* 


.01 


*v 


a lues at 


.00 percentile 


in Table 


9 





under the given percentile points, it became obvious that 
those obtained from lengths of 4000 had a more stable re- 
lationship with the x 2 distribution. The line of the plotted 
points is almost a straight line. The values for the extreme 



lengths of 500 and 6000 arc the most erratic, while those for 
lengths of 2000 and 1000 have only one and two points, re- 
spectively (out of eight percentile points), which deviate 
from a linear trend. 
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I mplic at Ion an d Po ssib le Exten sion 
of thi s Project ' — - 

By an empirical procedure , the author obtained 
the power of Darwin’s LRC at one point in the set of all 
possible alternative hypotheses. Pairs of sequences were 
generated from only one pair of sequences. If this pro- 
cedure were to be replicated for other pairs of sequences 
under different alternative hypotheses and for the same 
lengths, boundary points corresponding to the different 
alternative hypotheses would be obtained. Such replica- 
tion would give more information on the range of the 
boundary points. 



It should be recalled that the transition proba- 
bilities on which the simulation was based were calculated 
from a set of 16 interaction matrices of eighth-grade 
mathematics classes, split into two subsets of classes 
having no significant difference in the achievement means. 
If conditions such as grade level, subject taught, and 
educational criterion are varied from one alternative 
hypothesis to another, the simulation would produce 
approximate LRC and z values which would serve as cutoff 
points for tests on data sets which have conditions simi- 
lar to those characterising the basic data sets in the 
simulation process. 

However, even without the replications suggested 
here, the distribution of LRC and z values generated 
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from one alternative hypothesis can give the interaction 
analyst a fair idea of the magnitude of the LRC and z 



values in comparison with the tabulated theoretical dis- 
tribution of Chi Square. A user may not necessarily us>j 
the values presented in this paper, but, having some em- 
pirical values to compare his own results with, he may 
find in them some basis for his interpretation oJ: the 



significant results of Darwin * s LRC. 



TABLE 11 

SUGGESTED BOUNDARY POINTS FOR LRC AND z, BASED 
ON VALUES AT THE 85th AND 9Sth PERCENTILES 
OF THEIR EMPIRICAL DISTRIBUTIONS FOR 
DIFFERENT SEQUENCE LENGTHS 



Length of 
Sequence 


95th Percentile 


99th Percentile 


LRC 


z 


LRC 


i"? 


500 


100.74 


0.80 


102.27 


0.91 


1000 


131.02 


2.80 


140.27 


3.36 


2000 


168.75 


4.98 


179.48 


5.56 


4000 


247.41 


9.17 


261.38 


9.47 


6000 


312.75 


11.6 2 


327.12 


12.19 



Thus, the critical values of LRC and z were taken from 
Table 10 and presented in Tanle 11 as the suggested boundary 
points (at the 95th and 99th percentiles of their distributions) 
for the rejection of the hypothesis. It is left to the reader to 
use his insight and experience with his data to judge whether 
these values are relevant to his own research. 
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